Boosting Decision Trees
نویسندگان
چکیده
A new boosting algorithm of Freund and Schapire is used to improve the performance of decision trees which are constructed usin: the information ratio criterion of Quinlan’s C4.5 algorithm. This boosting algorithm iteratively constructs a series of decision tress, each decision tree being trained and pruned on examples that have been filtered by previously trained trees. Examples that have been incorrectly classified by the previous trees in the ensemble are resampled with higher probability to give a new probability distribution for the next ace in the ensemble to tnin on. Results from optical cha-xc:er reco~tion (OCR), and knowledge discovery and data mining problems show that in comparison to single trees, or to trees trained independenrly_ or to trees trained on subsets of the feature space, the boosring ensemble is much better.
منابع مشابه
Boosting with Multi-Way Branching in Decision Trees
It is known that decision tree learning can be viewed as a form of boosting. However, existing boosting theorems for decision tree learning allow only binary-branching trees and the generalization to multi-branching trees is not immediate. Practical decision tree algorithms, such as CART and C4.5, implement a trade-off between the number of branches and the improvement in tree quality as measur...
متن کاملA Boosting method in Combination with Decision Trees
This paper describes boosting – a method, which can improve results of classification algorithms. The use of this method aims at classification algorithms generating decision trees. A modification of the AdaBoost algorithm was implemented. Results of performance tests focused on the use of the boosting method on binary decision trees are presented. The minimum number of decision trees, which en...
متن کاملParallelizing Boosting and Bagging
Bagging and boosting are two general techniques for building predictors based on small samples from a dataset. We show that boosting can be parallelized, and then present performance results for parallelized bagging and boosting using OC1 decision trees and two standard datasets. The main results are that sample sizes limit achievable accuracy, regardless of computational time spent; that paral...
متن کاملAccelerated Gradient Boosting
Gradient tree boosting is a prediction algorithm that sequentially produces a model in the form of linear combinations of decision trees, by solving an infinite-dimensional optimization problem. We combine gradient boosting and Nesterov’s accelerated descent to design a new algorithm, which we call AGB (for Accelerated Gradient Boosting). Substantial numerical evidence is provided on both synth...
متن کاملBoosting bonsai trees for efficient features combination: application to speaker role identification
In this article, we tackle the problem of speaker role detection from broadcast news shows. In the literature, many proposed solutions are based on the combination of various features coming from acoustic, lexical and semantic information with a machine learning algorithm. Many previous studies mention the use of boosting over decision stumps to combine efficiently these features. In this work,...
متن کامل